Skip to content
This repository has been archived by the owner on Nov 24, 2023. It is now read-only.

syncer: don't skip jobs from same event when comparing table checkpoint #1752

Merged
merged 11 commits into from
Jun 17, 2021
Merged

syncer: don't skip jobs from same event when comparing table checkpoint #1752

merged 11 commits into from
Jun 17, 2021

Conversation

lance6716
Copy link
Collaborator

@lance6716 lance6716 commented Jun 5, 2021 •
edited
Loading

What problem does this PR solve?

close #1751

after all_mode paused, we could see

MySQL [all_mode]> select * from t2;
+----+-------+---------------------+
| id | name  | ts                  |
+----+-------+---------------------+
|  1 | Arya  | 2021-06-05 18:01:52 |
|  2 | Bran  | 2021-05-11 10:01:05 |
|  3 | Sansa | 2021-06-05 18:01:52 |
| 10 | 10    | 2021-06-05 18:02:02 |
| 20 | 20    | 2021-06-05 18:02:02 |
| 30 | 30    | 2021-06-05 18:02:02 |
+----+-------+---------------------+
6 rows in set (0.001 sec)

MySQL [all_mode]> select * from t1;
+----+---------+---------------------+---------------------+
| id | name    | dt                  | ts                  |
+----+---------+---------------------+---------------------+
|  1 | ar"ya   | 2021-06-05 18:01:52 | 2021-06-05 18:01:52 |
|  2 | catelyn | 2021-05-11 10:01:05 | 2021-05-11 10:01:05 |
|  3 | lalala  | NULL                | 2021-06-05 18:01:52 |
| 10 | 10      | NULL                | 2021-06-05 18:02:01 |
| 30 | 30      | NULL                | 2021-06-05 18:02:02 |
+----+---------+---------------------+---------------------+
5 rows in set (0.001 sec)

t1 losts a row with id = 20 of https://github.com/pingcap/dm/pull/1752/files#diff-5cc7189b51f6219ea0ce1c9920f92f84cee41b3fee093894a72ca5ff1c615b4aR2

What is changed and how it works?

Check List

Tests

  • Integration test

Code changes

  • Has interface methods change

Related changes

  • Need to cherry-pick to the release branch
  • Need to be included in the release note

@ti-chi-bot ti-chi-bot requested review from GMHDBJD and lichunzhu June 5, 2021 10:04
@lance6716 lance6716 added this to the v2.0.4 milestone Jun 7, 2021
@lance6716 lance6716 changed the title [WIP] add a test to reproduce BUG syncer: don't skip jobs from same event when comparing table checkpoint Jun 9, 2021
@lance6716 lance6716 added needs-cherry-pick-release-2.0 This PR should be cherry-picked to release-2.0. Remove this label after cherry-picked to release-2.0 needs-update-release-note This PR should be added into release notes. Remove this label once the release notes are updated labels Jun 9, 2021
@lance6716 lance6716 removed this from the v2.0.4 milestone Jun 9, 2021
@GMHDBJD GMHDBJD added this to the v2.0.4 milestone Jun 10, 2021
GMHDBJD
GMHDBJD approved these changes Jun 15, 2021
View reviewed changes
Copy link
Collaborator

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot added the status/LGT1 One reviewer already commented LGTM label Jun 15, 2021
Ehco1996
Ehco1996 reviewed Jun 15, 2021
View reviewed changes
Copy link
Contributor

@Ehco1996 Ehco1996 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM

syncer/checkpoint.go Show resolved Hide resolved
syncer/checkpoint.go Show resolved Hide resolved
syncer/checkpoint.go Show resolved Hide resolved
@ti-chi-bot
Copy link
Member

@Ehco1996: Thanks for your review. The bot only counts approvals from reviewers and higher roles in list, but you're still welcome to leave your comments.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • GMHDBJD
  • glorv

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added status/LGT2 Two reviewers already commented LGTM, ready for merge and removed status/LGT1 One reviewer already commented LGTM labels Jun 17, 2021
@glorv
Copy link
Collaborator

glorv commented Jun 17, 2021

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 4c80409

@ti-chi-bot ti-chi-bot merged commit 944c5aa into pingcap:master Jun 17, 2021
ti-chi-bot pushed a commit to ti-chi-bot/dm that referenced this pull request Jun 17, 2021
@lance6716 @ti-chi-bot
syncer: don't skip jobs from same event when comparing table checkpoi…
38c3c2e 
…nt (pingcap#1752)
@ti-chi-bot ti-chi-bot mentioned this pull request Jun 17, 2021
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #1781.

lance6716 added a commit to lance6716/dm that referenced this pull request Jun 17, 2021
@lance6716
syncer: don't skip jobs from same event when comparing table checkpoi…
44a6665 
…nt (pingcap#1752)
@lance6716 lance6716 mentioned this pull request Jun 17, 2021
Merged
lance6716 added a commit to lance6716/dm that referenced this pull request Jun 17, 2021
@lance6716
syncer: don't skip jobs from same event when comparing table checkpoi…
6cd65a9 
…nt (pingcap#1752)
ti-chi-bot added a commit that referenced this pull request Jun 17, 2021
@ti-chi-bot
syncer: don't skip jobs from same event when comparing table checkpoi…
e85bfd2 
…nt (#1752) (#1781)
GMHDBJD pushed a commit that referenced this pull request Jun 21, 2021
@lance6716
syncer: don't skip jobs from same event when comparing table checkpoi…
176f8e2 
…nt (#1752) (#1783)
GMHDBJD pushed a commit to GMHDBJD/dm that referenced this pull request Jun 21, 2021
@lance6716 @GMHDBJD
syncer: don't skip jobs from same event when comparing table checkpoi…
28a0f4d 
…nt (pingcap#1752) (pingcap#1783)
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@GMHDBJD GMHDBJD GMHDBJD approved these changes

@Ehco1996 Ehco1996 Ehco1996 approved these changes

@glorv glorv glorv approved these changes

@lichunzhu lichunzhu Awaiting requested review from lichunzhu

Assignees
No one assigned
Labels
needs-cherry-pick-release-2.0 This PR should be cherry-picked to release-2.0. Remove this label after cherry-picked to release-2.0 needs-update-release-note This PR should be added into release notes. Remove this label once the release notes are updated size/L status/can-merge status/LGT2 Two reviewers already commented LGTM, ready for merge
Projects
None yet
Milestone
v2.0.4
Development

Successfully merging this pull request may close these issues.

DM may lost data when resume from error in non GTID mode
5 participants
@lance6716 @ti-chi-bot @glorv @Ehco1996 @GMHDBJD